Shrinkage and Penalized Likelihood as Methods to Improve Predictive Accuracy
نویسندگان
چکیده
Hans C. van Houwelingen Saskia le Cessie Department of Medical Statistics, Leiden, The Netherlands P.O.Box 9604 2300 RC Leiden, The Netherlands email: [email protected] Abstract A review is given of shrinkage and penalization as tools to improve predictive accuracy of regression models. The James-Stein estimator is taken as starting point. Procedures covered are the Pre-test Estimation, Ridge Regression of Hoerl and Kennard, the Shrinkage Estimators of Copas and Van Houwelingen and Le Cessie, the LASSO of Tibshirani and the Garotte of Breiman. An attempt is made to place all these procedures in a unifying framework of semi-Bayesian methodology. Applications are briefly mentioned, but not amply discussed.
منابع مشابه
Comparison of Ordinal Response Modeling Methods like Decision Trees, Ordinal Forest and L1 Penalized Continuation Ratio Regression in High Dimensional Data
Background: Response variables in most medical and health-related research have an ordinal nature. Conventional modeling methods assume predictor variables to be independent, and consider a large number of samples (n) compared to the number of covariates (p). Therefore, it is not possible to use conventional models for high dimensional genetic data in which p > n. The present study compared th...
متن کاملApplication of shrinkage techniques in logistic regression analysis: a case study
Logistic regression analysis may well be used to develop a predictive model for a dichotomous medical outcome, such as short-term mortality. When the data set is small compared to the number of covariables studied, shrinkage techniques may improve predictions. We compared the performance of three variants of shrinkage techniques: 1) a linear shrinkage factor, which shrinks all coef®cients with ...
متن کاملComparison of Cox Model Methods in A Low-dimensional Setting with Few Events
Prognostic models based on survival data frequently make use of the Cox proportional hazards model. Developing reliable Cox models with few events relative to the number of predictors can be challenging, even in low-dimensional datasets, with a much larger number of observations than variables. In such a setting we examined the performance of methods used to estimate a Cox model, including (i) ...
متن کاملSNP Selection in Genome-Wide and Candidate Gene Studies via Penalized Logistic Regression
Penalized regression methods offer an attractive alternative to single marker testing in genetic association analysis. Penalized regression methods shrink down to zero the coefficient of markers that have little apparent effect on the trait of interest, resulting in a parsimonious subset of what we hope are true pertinent predictors. Here we explore the performance of penalization in selecting ...
متن کاملJackknifed Liu-type Estimator in Poisson Regression Model
The Liu estimator has consistently been demonstrated to be an attractive shrinkage method for reducing the effects of multicollinearity. The Poisson regression model is a well-known model in applications when the response variable consists of count data. However, it is known that multicollinearity negatively affects the variance of the maximum likelihood estimator (MLE) of the Poisson regressio...
متن کامل